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Abstract 



■ We present a comprehensive analysis of the sensitivity of the TESLA e + e collider to 

£S) | top flavour-changing neutral couplings to the Z boson and photon. We study single top 

production and top decay processes, and we consider the cases without beam polarization, 
with only e~ polarization and with e~ and e + polarization. We show that the use of the 
latter substantially enhances the sensitivity to discover or bound these vertices, and for 
some of the couplings the expected LHC limits could be improved by factors 2 — 14 for 
equal running times. 



1 Introduction 



^ It is generally believed that the top quark, because of its large mass, will be a sensitive probe 

into physics beyond the Standard Model (SM) pj. In particular, its couplings to the gauge 
and Higgs bosons may show deviations with respect to the SM predictions. In the SM the 
flavour-changing neutral (FCN) couplings Ztq, with q = u,c, vanish at tree-level by the GIM 
mechanism, and the yig and gtq ones are zero as a consequence of the unbroken SC/(3) c xU(1)q 
symmetry. The Htq couplings also vanish due to the existence of only one Higgs doublet. 
These types of vertices can be generated at the one-loop level, but they are very suppressed 
by the GIM mechanism, because the masses of the charge —1/3 quarks in the loop are small 
compared to the scale involved. The single top production branching may be estimated 
roughly by Br(Z — ► tc) = 1.5 x 10~ 13 ||, and the calculation of the branching ratios for top 
decays mediated by these FCN operators yields the SM predictions Br(t — ► Zc) = 1.3 x 10 -13 , 
Br(t -> yc) = 4.5 x 10" 13 , Br(i -» gc) = 3.5 x 10~ n g|, Br(t -> He) = 3.5 x 10~ 14 Q Q, 
and smaller values for the up quark. However, in many simple SM extensions these rates 
can be orders of magnitude larger. For instance, in models with exotic quarks Br(t — > Zq) 
can be of order 10~ 3 — 10~ 2 ||. Two Higgs doublet models allow for Br(t — ► Zc) = 10~ 6 , 
Br(i — * yc) = 10 -7 , Br(£ — * gc) = 10~ 4 |@], and in R parity-violating supersymmetric models 
one can have Br(t -> Zc) = 10~ 4 , Br(t yc) = 10" 5 , Br(t gc) = 10" 3 §. Top FCN 
decays into a light Higgs boson and an up or charm quark can also have similar or larger 
rates in models with exotic quarks j|, ||, with more than one Higgs doublet || ||] or with 
supersymmetry fTTJ]. Hence, top FCN couplings offer a good place to search for new physics, 



x We assume a Higgs mass Mh = 120 GeV. 
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which may manifest if these vertices are observed in future colliders. In addition, the study 
of FCN couplings provides model-independent information on the charged current couplings 
and the unitarity of the CKM matrix |]ll[ . Here we will focus on FCN interactions involving 
the top, a light charge 2/3 quark q and a neutral gauge boson V = Z, 7. At present the best 
limits on Ztq couplings come from LEP 2, Bi{t — > Zq) < 0.07 13], and the best limits on 
jtq couplings from Tevatron, Br(t — ► 75) < 0.032 fli~4j| . They are very weak but will improve 
in the next years, first with Tevatron Run II, and later with the next generation of colliders. 

The CERN LHC will be a top factory. With a tt production cross-section of 830 pb, at its 
100 fb _1 luminosity phase it will produce 8.3 x 10 top-antitop pairs per year. In addition, it 
will produce 3 x 10 7 single tops plus antitops via other processes [15, [TJ|. This makes LHC an 
excellent machine for the investigation of the top quark properties. The search for FCN top 
couplings can be carried out examining two different types of processes. On the one hand, 
we can look for rare top decays t —> Zq [17], t — ► 75 [p^j , t — » gq [19] or t — ► Hq |2C] of the 
tops or antitops produced in the SM process gg, qq — ► tt. On the other hand, one can search 
for single top production via an anomalous effective vertex: Zt and jt production [21|, the 



production of a top quark without or with a light jet [22, p3| ], and Ht production pQ] . In 
these cases the top quark is assumed to decay in the SM dominant mode t — * Wb. One can 
also search for like-sign tt production |^4j and other exotic processes. 

The TESLA e + e~ collider with a centre of mass (CM) energy of ^fs = 500 GeV has a 
tree-level tt production cross-section of 0.52 pb, and produces only 1.56 x 10 5 top-antitop 
pairs per year with its expected luminosity of 300 fb _1 . However, e + e~ colliders are cleaner 
than hadron colliders. For instance, the signal to background ratio S/B for rare top decays 
can be 7 times larger in TESLA than in LHC. But the sensitivity to rare top decays is given 
in the Gaussian statistics limit by the ratio S/y/B, and the larger LHC cross-sections make 
difficult for TESLA to compete with it in the search for anomalous top couplings. 

In this paper we show that the use of beam polarization in TESLA substantially enhances 
its sensitivity to discover or bound top anomalous FCN couplings and allows to improve some 
of the expected LHC limits up to an order of magnitude ||| . We first study the single top 
production process e + e~ — ► tq, mediated by Ztq or ■jtq anomalous couplings |26|]. Then we 
study rare top decays in the processes e + e~ — ► ti, with subsequent decay t — ► Vq. In all 
cases we take into account the charge conjugate processes as well: we sum tq + tq production, 
and we consider t — > Vq or t — ► Vq. Single top production and top decay processes are 
complementary. Single top production is more sensitive to top anomalous couplings but top 
decays can help to disentangle the type of anomalous coupling involved {Ztq or jtq) if a 
positive signal is discovered. 

We consider the planned CM energies of 500 and 800 GeV, and for both we analyse the 
cases: {i) without beam polarization, {ii) with 80% e~ polarization, and {Hi) with 80% 
e~ , 45% e + polarization. The paper is organized as follows. In Section |2| we describe the 
procedure used to compute the signals and backgrounds and to obtain the limits. In Section || 
we analyse single top production. In Sections || and || we consider top decays t — ► 79 and 
t — ► Zq, respectively. In Section |6| we summarize the results and draw our conclusions. 
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2 Generation of signals and backgrounds 



In order to describe the FCN couplings among the top, a light quark q and a Z boson or a 
photon A we use the Lagrangian 



X tq hM q PL + <Pn)^ + |^ n tq i(4 q - < 9 75)^ 
+eA t9 t(A^-A t a (?7 5) ! ^^, (.1) 



m t 

where -Pr,l = (l±75)/2. The chirality-dependent parts are normalized to (x^ q ) 2 + (x^ g ) 2 = 1, 



( K tq)' 2 + ( K tq) 2 = 1) Wg) 2 + (Kq) 2 = 1- This effective Lagrangian contains 7m terms of 
dimension 4 and terms of dimension 5. The couplings are constants corresponding to the 
first terms in the expansion in momenta. The terms are the only ones allowed by the 
unbroken gauge symmetry, SU(3) C x U(1)q. Due to their extra momentum factor they grow 
with the energy and make large colliders the best places to measure them. 

For single top production we study the process e + e~ — > tq mediated by Ztq or jtq 
anomalous couplings (see Fig. |l|). We will only take one anomalous coupling different from 
zero at the same time. However, if a positive signal is discovered, it may be difficult to 
distinguish only from this process whether the anomalous coupling involves the Z boson, the 
photon or both. On the other hand, in principle it could be possible to have a fine-tuned 
cancellation between Z and 7 contributions that led to a suppression of this signal. 




Figure 1: Feynman diagrams for e + e —* tq via Ztq or jtq FCN couplings. The top quark is 
off-shell and has the SM decay. 

For top decays we study the SM process e + e~ — > ti, followed by antitop decay mediated 
by an anomalous Ztq or 'ytq coupling (see Fig. |2|). This gives the signals tqZ and tq'y, and 
the observation of the final state distinguishes Ztq and ^ytq couplings. In the tq, tqZ and tqj 
signals the top is assumed to decay via t — > W + b — > l + vb, with I = e,fi. For the tqZ signal 
we only consider the Z boson decays to electrons and muons. 

For the tq signal we calculate the matrix element e + e~ — > tq — > W + bq — * l + ubq. For the 
tqZ and tq'y signals we calculate e + e~ — ► ti — ► W + bqZ — ► l + ubql' + l'~ and e + e~ — ► ti — ► 



W + bq r y — ► l + vbqj, respectively. These matrix elements are evaluated using HELAS [27] and 
introducing a new HELAS-like subroutine I0V2XX to compute the non-renormalizable 0^ 
vertex. This new routine has been checked by hand. In all cases we sum the contribution of 
the charge conjugate processes. For the tqV signals there is an additional contribution from 
tq production plus radiative emission of a Z boson or a photon. This correction is suppressed 
because it does not have the enhancement due to the i on-shell, and is even smaller after the 
kinematical cuts for the signal reconstruction. 
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Figure 2: Feynman diagrams for FCN antitop decay via Ztq or jtq FCN couplings. 

The background for the tq signal is given by W + qq' production with W + decay to electrons 
and muons. The leading contribution to this process is W + W~ production with W~ hadronic 
decay, but it is crucial for the correct evaluation of the background after kinematical cuts to 
take into account the 7 interfering Feynman diagrams for e + e~ — > W + qq' . Taking all the 
interfering diagrams for e + e~ — > l + uqq' into account does not make any appreciable change 
in the cross-section. The backgrounds for the tqZ and tq'j signals are analogous, W + qq'Z 
and W + qq' / y, with 46 and 44 diagrams, respectively. These three backgrounds are evaluated 
using MadGraph [£8| and modifying the code to include the W + decay. 

To simulate the calorimeter energy resolution we perform a Gaussian smearing of the 



charged lepton (I), photon (7) and jet (j) energies using a realistic calorimeter resolution [29] 
of 

AE 1 ^ 10% 1(V AE> 50% AM 

e 1% , -— = -t= e 4% , (2) 



E l >"! i/Ehi ' Ei ^/eJ 

where the energies are in GeV and the two terms are added in quadrature. For simplicity 
we assume that the energy smearing for muons is the same as for electrons. Note that more 
optimistic resolutions would improve our results. We then apply detector cuts on transverse 
momenta px and pseudorapidities r\ 

p T > 10 GeV , \rj\ < 2.5 . (3) 

The cut on pseudorapidity corresponds to a polar angle 10° < 9 < 170°. We reject the events 
in which the jets and/or leptons are not isolated, requiring that the distances in (77, 0) space 
AR satisfy AR > 0.4. We do not require specific trigger conditions, and we assume that the 
presence of high px charged leptons will suffice. 

After signal and background reconstruction, which will be analysed in detail for each of 
the processes discussed, we require a b tag on the jet associated to the decay of the top 
quark to reduce the backgrounds. We require the b tagged jet to have |%| < 2 (polar angle 
15° < Of, < 165°) and energy Ef, > 45 GeV. We assume a b tagging efficiency of 60%, and 
mistagging rates of 5% for charm and 0.5% for lighter quarks pop . These are average numbers 
appropriate for the Ef, kinematical distributions we will obtain later. 

After kinematical cuts, for each of the cases studied we obtain two types of limits on the 
anomalous coupling parameters X tg , Kt q , Xt q - Below we outline the procedure used. The 
correct statistical treatment of signals and backgrounds is specially necessary in our study 
since the backgrounds are very small, sometimes much less than one event even for high 
integrated luminosities. 

Assuming that no signal is observed after the experiment is done, i.e. the number of 
observed events no equals the expected background n^, we derive 95% confidence level (CL) 
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upper bounds on the number of events expected n s . We use the Feldman- Cousins construction 
for the confidence intervals of a Poisson variable |31] evaluated with the PCI package [32]. 

On the other hand, we can obtain the smallest value of n s such that a positive signal 
is expected to be observed with 3 a significance, assuming that the number of observed 
events for 3 a 'evidence' n e equals n s + rib- For a large number of background events, the 
Poisson probability distribution can be approximated by a Gaussian of mean and standard 
deviation \fn h . The requirement of 3 a significance is then simply n s /y/n b > 3. However, this 
is seldom the case for our study, where the backgrounds are very small. In such case, we use 
the estimator based on the V number (see for example |33[]). The number V(n) is defined 
as the probability of the background to fluctuate and give n or more observed events. n e is 
then defined as the smallest value of n such that 1 — V(n) > 99.73%, corresponding to three 
Gaussian standard deviations. 

Another possible estimator for the evidence of a signal can be built using Feldman-Cousins 
intervals. For a fixed value of nt>, we define the number of observed events for 3 a evidence n e 
as the smallest value of the number of observed events n such that the 99.73% CL Feldman- 
Cousins intervals do not contain zero. These two estimators for the evidence of a signal 
can be shown to be equivalent, and for tij, > 9 both give similar results to the Gaussian 
approximation n s /^/n b > 3. 

In our analysis we find that usually 3 a evidence limits are numerically larger than 95% 
upper limits, but this is not always the case. For very small backgrounds 3 a limits are 
smaller, what means that the potential to discover a new signal is better than the ability to 
set upper bounds on it if nothing is seen. This behaviour can exhibit fluctuations resulting 
from the discreteness of Poisson statistics, but in general and comparing with LHC, the 
TESLA discovery potential is better than the potential to set upper limits. 



3 Single top production e + e — ► tq 

The process e + e~ —* tq gives better limits on the top anomalous couplings than top de- 
cays. However, it has the disadvantage that the final state l + vbj does not distinguish the 
type of coupling involved. The background is W + jj production with W + — > l + v and a jet 
misidentified as a o. 

We take only one type of FCN couplings different from zero at the same time, and we 
evaluate three signals: (i) with Ztq 7^, (ii) with Ztq a^, and (in) with ytq couplings. Their 
cross-sections depend slightly on the chirality of the anomalous couplings. The chirality- 
dependent parts can be written as (v — 075), with v 2 + a 2 = 1. For definiteness, we set v = 1, 
a = in our evaluations. The results are the same setting v = and a = 1. For a CM energy 
of 500 GeV and the three polarization options discussed the cross-section for 7^ couplings 
differs =Fl% setting v = ±a = l/\/2, and for a^ u couplings it differs ±1.2%. 

The signals are reconstructed as follows. The neutrino momentum p u can be identified 
with the missing momentum of the event. The longitudinal missing momentum can also be 
used, and p u is reconstructed without any ambiguity. The W + momentum is then the sum 
of the momenta of the charged lepton and the neutrino. In the case of tq production, the 
invariant mass of the W + and one of the jets, m£ ec , is consistent with the top mass, and the 
other jet has an energy E q around = (s — mj)/(2y / s). Of the two possible pairings, we 
choose the one minimizing (m\ ec — mt) 2 + (Eg — E®) 2 and require a b tag on the jet associated 
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to the top quark. The kinematical distributions for m£ ec and E q for the signals and the 
background at a CM energy of 500 GeV without beam polarization are plotted in Figs. || and 
|j. The Eb distribution is shown in Fig. ||. 

Another interesting variable to distinguish the signals from the background is the two- 
jet invariant mass MS^L. The W + jj background is dominated by W + W~ production with 
W~ — > jj, and the M^ c _ distribution peaks around Myy, as can be clearly seen in Fig. ||. 
A veto cut on My^L can eliminate a large fraction of the background but makes compulsory 
to calculate correctly the cross-section to include all the diagrams for e + e~ — > W + qq' . Also 
of interest are the total transverse energy Ht and the charged lepton energy E\ in Figs. [?] 
and |8[ The kinematical distributions with polarized beams are very similar except the Ht 
distribution. In this case polarization decreases the peak of the background around Ht = 200. 

To enhance the signal significance we perform kinematical cuts on these variables. How- 
ever, we find that the veto cut on M^L is unnecessary in single top production since the 
requirement Ef, > 45 GeV and the kinematical cut on m£ ec practically eliminate the peak 
in the M^L distribution. A cut on E q is unnecessary because this variable is kinematically 
related to m]i ec , and we prefer to apply a cut on m\ cc to show the presence of a top quark 
in the signal. For simplicity, we choose to apply the same cuts for the three signals and the 
three polarization options, but different for CM energies of 500 and 800 GeV. We choose the 
cuts trying to maintain the independence of the cross-section on the chirality of the coupling. 
Obviously, our results could be improved modifying the cuts for each type of coupling and 
each polarization option. We now discuss the results for 500 GeV and 800 GeV in turn. 

3.1 Limits at 500 GeV 

The kinematical cuts for 500 GeV are collected in Table |], and the cross-sections before 
and after cuts for the three polarization options in Table [|. We normalize the signals to 
X tq = 0.06, n tq = 0.02, \ tq = 0.02 and sum the charge conjugate processes. For the chiralities 
with v = dbo the cross-section after cuts differs ±6.8% for couplings and =f4.6% for a^ u 
couplings. 

Variable 500 GeV cut 
mf c 160-190 
H T > 220 

Ei < 160 

Table 1: Kinematical cuts for the three tq signals and the three polarization options at a CM 
energy of 500 GeV. The masses and the energies are in GeV. 

We notice in Table § the usefulness of polarization: e~ polarization decreases the back- 
ground by a factor of 5, without affecting too much the signals. e + polarization further 
decreases the background and even increases the cross-section of the signals with respect to 
the values without polarization. 

We express the limits on the anomalous couplings in terms of top decay branching ratios, 
using Tt = 1.56 GeV. As explained in the previous Section, we obtain 95% CL upper limits for 
the case that nothing is observed and 3er discovery limits. Since the number of background 
events is small, the limits do not scale with the luminosity L as 1/yL- In Table ^ we 
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No pol. 


rOl. 


6 


rOl. 


6 €~^~ 


before 


after 


before 


after 


before 


after 


cuts 


cuts 


cuts 


cuts 


cuts 


cuts 


0.183 


0.137 


0.162 


0.121 


0.215 


0.161 


0.199 


0.153 


0.176 


0.135 


0.234 


0.179 


0.375 


0.288 


0.375 


0.287 


0.510 


0.391 


19.5 


0.0734 


4.06 


0.0154 


2.40 


0.0092 



tq + tq (Z,Jn) 
tq + tq (Z, a^y) 
tq + tq (7) 
W ± jj 

Table 2: Cross-sections (in fb) before and after the kinematical cuts in Table || for the three tq 
signals and their background at a CM energy of 500 GeV, for the three polarization options. 
We include b tagging efficiencies and use X tq = 0.06, Kt q = 0.02, X tq = 0.02. 



quote limits for a reference integrated luminosity of 100 fb -1 for comparison with other 
processes, and in Table |3| for 300 fb -1 , corresponding to one year of operation with the 
expected luminosity. 

No pol. 
95% 3ct 
Br(i -> Zq) (7^) 7.9 x 1CT 4 1.2 x 1CT 3 
Br(i -> Zq) (a^) 6.3 x 10~ 5 9.4 x 1(T 5 
Br(< -> 79) 3.9 x 1(T 5 5.9 x 10~ 5 

Table 3: Limits on top FCN decays obtained from single top production at 500 GeV with a 
reference luminosity of 100 fb -1 for the three polarization options. 
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e 




Pol. 


e e + 




95% 


3(7 




95% 


3(7 




1 x icr 4 


7.5 x 10- 


-4 


4.4 x 10~ 4 


4.2 x 10 


-4 


7 x icr 5 


6.0 x 10- 


■5 


3.5 x 10~ 5 


3.4 x 10 


-5 


2 x 1(T 5 


3.3 x 10" 


■5 


1.9 x 10~ 5 


1.8 x 10 


-5 



No pol. 

95% 3(T 
Br(t -> Zq) (7^) 4.4 x 10" 4 6.1 x 10~ 4 
Br(t — ^ Zq) (a^) 3.5 x lO" 5 4.8 x lO" 5 
Br(t -> 73) 2.2 x 10~ 5 3.0 x 10~ 5 

Table 4: Limits on top FCN decays obtained from single top production at 500 GeV with a 
luminosity of 300 fb -1 for the three polarization options. 



Pol. 


e 




Pol. 


e e + 




95% 


3<7 




95% 


3a 




1 x 10" 4 


3.9 x 10- 


-4 


1.9 x 10~ 4 


2.2 x 10" 


-4 


4 x 10~ 5 


3.1 x 10- 


-5 


1.5 x 10~ 5 


1.7 x 10- 


-5 


3 x 10~ 5 


1.7 x 10- 


-5 


8.2 x 10~ 6 


9.3 x 10" 


-6 



3.2 Limits at 800 GeV 

We write the kinematical cuts for 800 GeV in Table || and the cross-sections before and after 
cuts in in Table |6|. We normalize the signals to X tq = 0.06, Kt q = 0.02, X tq = 0.02 and sum 
the charge conjugate processes. The signal cross-sections with non-renormalizable couplings 
do not decrease going from 500 to 800 GeV, whereas the background decreases by a factor of 
2.3. This improves the sensitivity for cTuv couplings with respect to 500 GeV. Unfortunately, 
the signal with 7^ couplings also decreases, and thus the results are worse in this case. Again 
we observe the usefulness of polarization: using only e~ polarization reduces the background 
5 times and using e + polarization as well reduces it 8 times. In Table we gather the limits 
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for a reference integrated luminosity of 100 fb and in Table || for 500 fb , collected in one 
year with the expected luminosity. 



Variable 800 GeV cut 



m\ ec 


160-190 




> 450 


Ei 


< 300 



Table 5: Kinematical cuts for the three tq signals and the three polarization options at a CM 
energy of 800 GeV. The masses and the energies are in GeV. 







No pol. 


Pol. 


e 


Pol. 


e e + 






before 
cuts 


after 
cuts 


before 
cuts 


after 
cuts 


before 
cuts 


after 
cuts 


tq + tq 


(Z,7„) 


0.0776 


0.0498 


0.0684 


0.0440 


0.0912 


0.0586 


tq + iq 




0.198 


0.149 


0.175 


0.132 


0.233 


0.175 


tq + tq 


(7) 


0.389 


0.293 


0.389 


0.293 


0.528 


0.398 


W ± jj 




8.45 


0.0125 


1.75 


0.0028 


1.03 


0.0018 



Table 6: Cross-sections (in fb) before and after the kinematical cuts in Table [| for the three tq 
signals and their background at a CM energy of 800 GeV, for the three polarization options. 
We include b tagging efficiencies and use X tq = 0.06, Kt q = 0.02, \ tq = 0.02. 



No pol. 

95% 3 cr 

Br(t -> Zq) (7 M ) 1.3 x 1CT 3 1.6 x 1(T 3 
Br(t Zq) (cv) 3.9 x 1CT 5 4.7 x 10 
Br(t -> >yq) 2.3 x 1(T 5 2.8 x 10 



—5 
-5 



Pol. 


e 




Pol. 


e e + 






95% 


3ct 




95% 


3 


a 




1.1 x 10" 3 


1.4 x 10 


-3 


8.3 x 10~ 4 


8.0 x 


10 


-4 


3.2 x 10~ 5 


4.2 x 10 


-5 


2.5 x 10~ 5 


2.4 x 


10 


-5 


1.7 x 10~ 5 


2.2 x 10 


-5 


1.3 x 10~ 5 


1.2 x 


10 


-5 



Table 7: Limits on top FCN decays obtained from single top production at 800 GeV with a 
reference luminosity of 100 fb -1 for the three polarization options. 



No pol. Pol. e~ Pol. e~ e+ 

95% 3cr 95% 3ct 95% 3 a 

Br(f -» Zq) (7^) 4.4 x 10" 4 5.9 x 10~ 4 2.9 x 10~ 4 4.3 x 10" 4 2.4 x 10~ 4 2.3 x 10" 4 

Br(t -> Zq) (a^) 1.3 x 10~ 5 1.7 x 10" 5 8.6 x 10~ 6 1.3 x 10~ 5 6.2 x 10" 6 7.0 x 10~ 6 

Br(t^ 7 g) 7.8 xlO" 6 1.0 x 10~ 5 4.5 x 10" 6 6.7 x 10~ 6 3.7 x 10" 6 3.6 x 10" 6 

Table 8: Limits on top FCN decays obtained from single top production at 800 GeV with a 
luminosity of 500 fb -1 for the three polarization options. 



For integrated luminosities of 100 fb the limits on branching ratios of decays mediated 
by non-renormalizable couplings are 1.5 times better than at 500 GeV, whereas the other 
limits are worse. Comparing the limits for 100 fb -1 and 500 fb -1 we notice that in the cases 
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with polarization the limits for 500 fb _1 are a factor of 3.3 — 3.8 smaller instead of ^/5 ~ 2.2. 
This improvement beyond the 1/y/L scaling is a consequence of the small backgrounds in 
these cases. 



4 Process e + e — ► tqj 

We begin the analysis of top decay processes with the more interesting case of the tqj signal. 
We study tt production with i — > and t decay to W + b, that gives a final state l + vbj^f, and 
sum the charge conjugate process. For equal values of Xt q , the process has a smaller cross- 
section than single top production. The main reasons are: (i) top decays are insensitive to 
the momentum factor q u of the coupling, (ii) the phase space for the production of a 
top-antitop pair is smaller. Nevertheless, this process can be useful to determine the nature 
of a FCN coupling involving the top quark, since the final state signals a "ftq coupling. The 
background is W + jj^f production with W + — > l + u and a jet misidentified as a b. As for single 
top production, we give our results for XJ q = 1, Xf q = but check that for other values of these 
parameters the differences are of order 0.1% and smaller that the Monte Carlo uncertainty 
for the three polarization options, before and after kinematical cuts. 

The tq'j signal can be reconstructed in a similar way as tq. The W + momentum is the 
charged lepton momentum plus the missing momentum. The invariant mass of the W + and 
one of the jets, m\ ec , is consistent with the top mass, and the invariant mass of the photon 
and the other jet, m r f c , is also consistent with nit- Of the two possible assignments, we 
choose the one minimizing {m\ ec — rat) 1 + (mf ec — rrit) 2 and require a b tag on the jet that 



corresponds to the top quark. We plot the distributions for both variables in Figs. |9| and 10, 
for a CM energy of 500 GeV without polarization. In the polarized case the distributions 
are indistinguishable. We note that the m Y f c distribution is more concentrated around mt 
because the energy resolution effects are less important. The different behaviour of the m\ ec 
distribution of the background around mt is related to the cut Eb > 45. It is very useful to 
consider also the invariant mass of the two jets, MJj^L, which is Mw for the background (see 



Fig. 11). We discuss the results for 500 GeV and 800 GeV independently. 



4.1 Limits at 500 GeV 

We write the kinematical cuts for 500 GeV in Table ||. The cut on mp c is more strict than 
the one on m\ ec because the reconstruction of the antitop mass is better. The cross-sections 
before and after cuts are gathered in Table [n]. Note that we normalize the signal to \t q = 0.04 
instead of Xtq = 0.02 as in the previous Section because the cross-sections are much smaller. 

500 GeV cut 
150-200 
160-190 
< 65 or > 95 

Table 9: Kinematical cuts for the tq^y signal and the three polarization options at a CM 
energy of 500 GeV. The masses are in GeV. 



Variable 



H 

,rcc 
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No pol. Pol. e~ Pol. e~ e+ 

before after before after before after 
cuts cuts cuts cuts cuts cuts 

trr + trr °- 0745 °- 0631 °- 0515 °- 0429 °- 0653 °- 0543 

W ± jjj 0.639 0.0014 0.144 3.1 x 10~ 4 0.0956 2.0 x 10~ 4 

Table 10: Cross-sections (in fb) before and after the kinematical cuts in Table [9| for the tqj 
signal and background at a CM energy of 500 GeV, for the three polarization options. We 
include b tagging efficiencies and use \t q = 0.04. 



The use of polarization is not as useful as for single top production. Although it reduces 
the W + jj^f cross-section up to a factor of 6, this background is already tiny without polar- 
ization, and there is little advantage in reducing it further. Moreover, the signal cross-section 
also decreases, and the limits obtained are in some cases worse (see Tables O and pi). The 



limits from the tqj signal are in all cases worse than those obtained from single top produc- 
tion. Polarization of e~ only gives worse results, but e~ and e + polarization improves the 
3<T discovery limits. 

No pol. Pol. e~ Pol. e~ e+ 

95% 3cr 95% 3cr 95% 3cr 

Br(t -> jq) 3.3 x 1CT 4 3.2 x 1CT 4 5.0 x 1CT 4 3.2 x 1CT 4 4.0 x 10~ 4 2.6 x 10~ 4 

Table 11: Limits on top FCN decays obtained from the tq'j process at 500 GeV with a 
reference luminosity of 100 fb -1 for the three polarization options. 



No pol. Pol. e" Pol. e~ e+ 

95% 3cr 95% 3 a 95% 3 a 

Br(t-> 79 ) 9.9xl0~ 5 1.3 x 10~ 4 1.6 x 10~ 4 1.6 x 10~ 4 1.3 x 10~ 4 8.3 x 10~ 5 

Table 12: Limits on top FCN decays obtained from the tq'j process at 500 GeV with a 
luminosity of 300 fb -1 for the three polarization options. 



4.2 Limits at 800 GeV 



For 800 GeV we perform the loose cuts for the top, antitop and W reconstruction in Table 13 



because the background is smaller than at 500 GeV. The signal cross-section also decreases 
in spite of the q v factor in the a^y coupling, and the limits obtained are worse. The cross- 
sections before and after cuts can be read in Table 14, and the limits obtained for 100 fb -1 
and 500 fb -1 in Tables 15 and [l6| respectively. The same comments as for 500 GeV apply in 
this case. We can see that the limits for 800 GeV and 500 fb -1 are similar but worse than 
those obtained for 500 GeV and 300 fb -1 . 
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800 GeV cut 
130-220 
150-200 
< 60 or > 100 

Table 13: Kinematical cuts for the tqj signal and the three polarization options at a CM 
energy of 800 GeV. The masses are in GeV. 



Variable 

m\ ec 

m rec 

M rec 
lvl w- 



tqj + tqry 
W ± jjj 



No pol. Pol. e~ 

before after before after 
cuts cuts cuts cuts 

0.0350 0.0327 0.0246 0.0227 

0.437 8.2 x 10" 4 0.0959 1.8 x 10" 4 



before 
cuts 

0.0314 

0.0613 



Pol. e~ e+ 

after 
cuts 

0.0289 

1.1 x 10" 4 



Table 14: Cross-sections (in fb) before and after the kinematical cuts in Table 13 for the tqj 
signal and background at a CM energy of 800 GeV, for the three polarization options. We 
include b tagging efficiencies and use Xt q = 0.04. 



No pol. Pol. e~ Pol. e~ e+ 

95% 3cr 95% 3 a 95% 3 a 

Br(t -> jq) 6.5 x 1CT 4 6.3 x 1(T 4 9.5 x 1CT 4 6.1 x 1CT 4 7.5 x 1CT 4 4.8 x 10~ 4 

Table 15: Limits on top FCN decays obtained from the tq'j process at 800 GeV with a 
reference luminosity of 100 fb" 1 for the three polarization options. 



No pol. Pol. e~ Pol. e~ e+ 

95% 3cr 95% 3cr 95% 3 a 

Br(t -> jq) 1.2 x 1CT 4 1.5 x 1(T 4 1.9 x 1CT 4 1.8 x 1CT 4 1.5 x 1(T 4 9.4 x 10" 5 

Table 16: Limits on top FCN decays obtained from the tq'j process at 800 GeV with a 
luminosity of 500 fb" 1 for the three polarization options. 



5 Process e + e — ► tqZ 

In this Section we study ti production with t — ► Zq and t — ► W + b, that gives a final state 
l + vbql' + l'~ . This signal is analogous to tq^y, but with the disadvantage that the partial 
width Br(Z — * = 0.067 considerably decreases the cross-sections for the signal and 

background. Comparing with single top production, we find that for equal values of X tq , Kt g 
the cross-section for tqZ is much smaller, mainly for the inclusion of the Z partial width, 
and for the smaller phase space also. In addition, for a^ v couplings the tqZ signal does not 
have an enhancement from the q v factor in the vertex. We give our results for 7^ and 
couplings using for definiteness x\ q = x^ q and K^ q = 1, n1 g = 0, respectively. We check that 
the differences with other chiralities are insignificant before and after kinematical cuts for the 
three polarization options. 

The background is W + jjZ production with W + — > Z — > and a b mistag. We 
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reconstruct the signal and background as follows. Of the two positively charged leptons, one 
results from the W + decay and it has with the neutrino (reconstructed from the missing 
momentum) an invariant mass M^ c + consistent with M^y. The other one and the negative 
charge lepton have an invariant mass M^ c close to Mz- If the two positive leptons have 
different flavours the assignment is straightforward, but if not we choose the pairing that 
minimizes (M^ c + — My/) 2 + (M^ ec — Mz) 2 - Then, we reconstruct the top and antitop masses 
as for the tq-f signal replacing the photon momentum with the Z momentum. The W~ 
reconstruction for the background is also similar. These distributions are plotted in Figs. 
0. 

5.1 Limits at 500 GeV 

Since the background is so small for this signal, we only apply very loose kinematical cuts 
for the top, antitop and W~ reconstruction. These can be read in Table [L7], and the cross- 
sections before and after these cuts in Table [l^. Note that we use a different normalization, 
X tq = 0.2, Kfg = 0.2, because the cross-sections are very small. 

500/800 GeV cut 
130-220 
150-200 
< 70 or > 90 

Table 17: Kinematical cuts for the tqZ signal and the three polarization options at CM 
energies of 500 and 800 GeV. The masses are in GeV. 



Variable 



mi 
M rec 



No pol. 



Pol. e 



tqZ + tqZ (7^) 
tqZ + tqZ (a^) 
W±jjZ 



before 
cuts 

0.114 

0.0877 

0.0059 



after 
cuts 

0.105 

0.0809 

1.0 x 10 _ 



before 
cuts 

0.0784 

0.0604 

0.0013 



after 
cuts 

0.0720 

0.0555 

2.4 x 10~ 5 



Pol. 

before 
cuts 

0.0995 

0.0766 

8.9 x 10~ 4 



after 
cuts 

0.0912 

0.0703 

1.6 x 10~ 5 



Table 18: Cross-sections (in fb) before and after the kinematical cuts in Table 17 for the tqZ 
signal and background at a CM energy of 500 GeV, for the three polarization options. We 
include b tagging efficiencies and use X tq = 0.2, Kt q = 0.2. 



Polarization is not as useful as for tq production, and the behaviour is similar as for the 
tq'y signal. This is reflected in the limits in Tables and where we find that polarization 
in some cases gives worse results. Note that 3<r discovery limits do not follow the same 
pattern for 100 fb -1 and 300 fb -1 due to the discreteness of Poisson statistics. These limits 
are much worse than those obtained from tq production, one order of magnitude worse for 
7 M couplings and two orders for a^ u couplings. In fact, this process would only be useful 
if a FCN top decay is detected with Br(t — ► Zq) ~ 10~ 3 . In such case, it would help to 
determine the nature of the top anomalous coupling. Besides, it is interesting to notice that 
the limits for 7^ and a^ iu couplings are remarkably similar. This confirms that this process 
is not sensitive to the q u factor of the a^ v vertex. 
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No pol. Pol. e~ Pol. e" e+ 

95% 3cr 95% 3 a 95% 3<j 

Br(< -> Zq) ( 7/i ) 5.4 x 10~ 3 3.5 x 1CT 3 8.0 x 1CT 3 2.6 x 10~ 3 6.3 x 1(T 3 2.0 x 10~ 3 

Br(t -> Zq) (o>) 5.7 x 10~ 3 3.7 x 10~ 3 8.3 x 10~ 3 2.7 x 10~ 3 6.5 x 10" 3 2.1 x 10~ 3 

Table 19: Limits on top FCN decays obtained from the tqZ process at 500 GeV with a 
reference luminosity of 100 fb -1 for the three polarization options. 



No pol. Pol. e~ Pol. e~ e+ 

95% 3ct 95% 3cr 95% 3cr 

Br(< -> Zq) ( 7m ) 1.8 x 10~ 3 1.2 x 10~ 3 2.7 x 10~ 3 1.7 x 10~ 3 2.1 x 10~ 3 1.4 x 10~ 3 

Br(t^ Zq) (a^) 1.9 x 10~ 3 1.2 x 10" 3 2.8 x 10" 3 1.8 x 10" 3 2.2 x 10" 3 1.4 x 10~ 3 

Table 20: Limits on top FCN decays obtained from the tqZ process at 500 GeV with a 
luminosity of 300 fb" 1 for the three polarization options. 



5.2 Limits at 800 GeV 



For 800 GeV we use the same set of cuts in Table |17J for the top, antitop and W reconstruc- 
tion, and obtain the cross-sections in Table 21. We notice that the background before cuts 



is larger than at 500 GeV (remember that it is dominated by W + W~ Z production, and its 
cross-section increases until CM energies around 900 GeV), but after the veto cut to remove 
events with on-shell W~ it becomes smaller as expected. We collect the limits obtained in 



Tables 22 and 23, The same comments as for the 500 GeV analysis apply. 



No pol. 



Pol. e" 



Pol. 



tqZ + tqZ (7^) 



tqZ + tqZ 
W ± jjZ 



(<v) 



before 
cuts 

0.0523 

0.0387 

0.0091 



after 
cuts 

0.0496 

0.0367 

2.3 x 1(T 5 



before 
cuts 



after 
cuts 



0.0367 0.0345 
0.0272 0.0255 
0.0020 5.0 x 1(T 



before 
cuts 

0.0467 

0.0346 

0.0012 



after 
cuts 

0.0439 

0.0325 

3.3 x 1(T 6 



Table 21: Cross-sections (in fb) before and after the kinematical cuts in Table 17 for the tqZ 
signal and background at a CM energy of 800 GeV, for the three polarization options. We 
include b tagging efficiencies and use X tq = 0.04, K tq = 0.04. 



No pol. Pol. e~ Pol. e~ e+ 

95% 3ct 95% 3cr 95% 3cr 

Br(t -> Zq) ( 7m ) 1.2 x 10" 2 3.7 x 10" 3 1.7 x 10" 2 5.4 x 10" 3 1.3 x 10" 2 4.2 x 10" 3 

Br(i -> Zq) (a MV ) 1.3 x 10" 2 4.0 x 10" 3 1.8 x 10" 2 5.8 x 10" 3 1.4 x 10" 2 4.6 x 10" 3 

Table 22: Limits on top FCN decays obtained from the tqZ process at 800 GeV with a 
reference luminosity of 100 fb -1 for the three polarization options. 
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No pol. Pol. e~ Pol. e" e+ 

95% 3cr 95% 3cr 95% 3cr 

Br(i -> Zq) (7^) 2.3 x 1(T 3 1.5 x 1CT 3 3.3 x 10~ 3 1.1 x 10~ 3 2.6 x 1(T 3 8.4 x 1(T 4 

Br(i -> Zq) (o>) 2.5 x 10~ 3 1.6 x 1CT 3 3.6 x 1(T 3 1.2 x 10~ 3 2.8 x 1(T 3 9.1 x 10 -4 

Table 23: Limits on top FCN decays obtained from the tqZ process at 800 GeV with a 
luminosity of 500 fb -1 for the three polarization options. 



6 Summary 

We have studied the most important signals of top FCN couplings to the Z boson and the 
photon that can be observed at a future e + e~ collider like TESLA. These are single top 
production e + e~ — ► tq, and rare top decays e + e~ — ► tt — > tqj, e + e~ — ► ti — ► tqZ. We have 
discussed three beam polarization options: no polarization, 80% e~ polarization and 80% e~ , 
45% e + polarization, for the two planned energies of 500 GeV and 800 GeV. In the following 
we summarize the differences among the signals and the influence of polarization and CM 
energy. 

Single top production versus top decays. Top decay signals are cleaner than single top 
production. This can be understood since the top decay signals W + bjV (V = Z, 7) have 
the enhancement over their background W + jjV of two on-shell particles, the top and the 
antitop, whereas single top production has only the enhancement due to the top on-shell and 
the a^ u coupling if that is the case. For instance, we can compare the values after kinematical 
cuts for 500 GeV without polarization for *ytq couplings. We see that the S/B ratio for the 
tq'j signal (after rescaling to Xt q = 0.02 as was assumed for single top production) is equal 
tol2, and for tq it equals 4. 

On the other hand, the cross-section for single top production is larger than for top 
decays for equal values of the parameters. For our previous example, a(tq + iq) = 0.288 fb, 
<r(i(/7 + iqj) = 0.0158 fb. The reasons are: (i) tq production is enhanced by the q v factor 
of the o^ v vertex, whereas tq'j is not; (tt) Phase space for the production of a tt pair is 
smaller than for tq. This makes the limits from tq production 4 times better for an integrated 
luminosity of 300 fb -1 . However, if a positive signal of a Vtq coupling is discovered through 
single top production, top decays can help to determine the nature of the coupling involved, 
i.e. whether it involves the photon or the Z boson, if Br(i — > Vq) ~ 10 -4 or larger. 

For Ztq couplings similar comments apply. The top decay signals are cleaner, especially 
for 7^ couplings, but the cross-sections are much smaller due to the leptonic partial width 
of the Z, Br(Z — ► = 0.067. The limits obtained for 7^ couplings are one order 

of magnitude worse, and those for a^ v couplings two orders of magnitude. Thus, the tqZ 
process is useful only if a signal with Br(t — ► Vq) ~ 10 -3 is detected. 

Influence of beam polarization. Polarization is very useful to improve the limits from 
single top production. In Table [2] we can observe that for a CM energy of 500 GeV the use 
of 80% e~ polarization decreases the background by a factor of 4.8 while keeping 90% of the 
signal, and additional e + polarization of 45% decreases the background by a factor of 8.1 and 
increases the signal 17% with respect to the values without polarization. The effect is similar 
at 800 GeV. e~ , e + polarization improves the 3<r discovery limits on 7^ couplings at 500 GeV 
with 300 fb -1 by a factor of 3, and the 3cr limits on a^ u couplings at 800 GeV with 500 fb -1 
by a factor of 2.6. The luminosities required to obtain the same results without polarization 
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would be 2100 fbr 1 and 3000 fb -1 , respectively. 

For top decay signals polarization is not as useful, because the backgrounds are already 
very small for unpolarized beams, and the luminosities required to glimpse the potential 
improvement would exceed 1000 fb -1 . In addition, the signal cross-sections decrease 10 — 
20%, in contrast to single top production. However, e~ and e + polarization still gives an 
improvement in the *ytq coupling 3cr discovery limits at 500 GeV with 300 fb -1 of a factor of 
1.6. This would be equivalent to double the luminosity without polarization. 

Influence of centre of mass energy. The increase in CM energy from 500 GeV to 800 
GeV enhances the sensitivity of single top production to a^ v couplings. This is because 
the signal cross-sections do not decrease (for the photon it even increases slightly) whereas 
the background is less than one half at 800 GeV. An e + e~ collider with 800 GeV and a 
reference luminosity of 100 fb"" 1 is sensitive to top rare decays mediated by these vertices 
with branching ratios 1.5 — 2 times smaller than one with 500 GeV and the same luminosity. 
Of course, the higher luminosity at 800 GeV has also to be taken into account, and then this 
energy is best suited to perform searches for these vertices. 

For normalizable 7^ couplings the signal cross-sections decrease for 800 GeV as expected, 
and thus the sensitivity is worse, even after taking into account the luminosity increase. More 
surprisingly, in top decays the limits are worse for the three types of couplings, because top 
decays are not sensitive to the q u factor of the vertex. Hence, to search for 7^ couplings 
in single top production and for all FCN coupling searches in top decays the CM energy of 
500 GeV is more adequate and gives the best results. 

Conclusions. We compare the best limits on anomalous couplings that can be obtained 
at TESLA and LHC. To obtain the values for LHC we rescale the data from the literature to 
a b tagging efficiency of 50% and keep the average mistagging rate used of 1% for other jets, 
which is somewhat optimistic. The best LHC limits on Vtc couplings come from top decays, 
whereas the best ones on Vtu couplings are from single top production. The LHC limit on 
Br(t — ► Zc) with couplings is estimated to be similar to the one with 7^ couplings having 
in mind the similar result observed in Section ||. We assume one year of running time in 
all the cases, that is, 100 fb" 1 for LHC, 300 fb" 1 for TESLA at 500 GeV and 500 fb" 1 for 
TESLA at 800 GeV. We use the statistical estimators explained in Section |2[ 

LHC TESLA 







95% 




3a 




95% 




3(7 




Br(t 




6.2 x 10" 


-5 


8.0 x 10" 


-5 


1.9 x 10- 


-4 


2.2 x 10- 


-4 


Br(t -» 


Zc) ( 7m ) 


7.1 x 10" 


-5 


1.0 x 10" 


-4 


1.9 x 10" 


-4 


2.2 x 10- 


-4 


Br(t -> 


Zu) (a^ u ) 


1.8 x 10" 


-5 


2.3 x 10" 


-5 


6.2 x 10- 


-6 


7.0 x 10- 


-6 


Br(t -» 


Zc) (07^) 


7.1 x 10" 


-5 


1.0 x 10" 


-4 


6.2 x 10- 


-6 


7.0 x 10- 


-6 


Br(t 


-> 7u) 


2.3 x 10- 


-6 


3.0 x 10" 


-6 


3.7 x 10" 


-6 


3.6 x 10" 


-6 


Br(t 


-» 7c) 


7.7 x 10- 


-6 


1.2 x 10- 


-5 


3.7 x 10- 


-6 


3.6 x 10- 


-6 



Table 24: Best limits on top FCN couplings that can be obtained at LHC and TESLA for 
one year of operation. 

We see that LHC and TESLA complement each other in the search for top FCN vertices. 
The 7„ couplings to the Z boson can be best measured or bound at LHC, whereas the 
sensitivity to the ones is better at TESLA. For photon vertices, LHC is better for ^ytu 
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and TESLA for jtc. The complementarity of LHC and TESLA also stems from the fact that 
TESLA will not be able to distinguish Ztq and jtq couplings in the limit of its sensitivity, 
whereas LHC will because final states are different and distinguish between them. On the 
other hand, the good charm tagging efficiency expected at TESLA will be able to distinguish 
Vtu and Vtc couplings looking at the flavour of the final state jet, what is more difficult to 
do at LHC. 
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Figure 3: Reconstructed top mass m\ cc distribution before kinematical cuts for the three tq 
signals and W + jj background at a CM energy of 500 GeV, without beam polarization. The 
cross-sections are normalized to unity. 
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Figure 4: E q distribution before kinematical cuts for the three tq signals and W + jj back- 
ground at a CM energy of 500 GeV, without beam polarization. The cross-sections are 
normalized to unity. 
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Figure 5: distribution before kinematical cuts for the three tq signals and W + jj back- 
ground at a CM energy of 500 GeV, without beam polarization. The cross-sections are 
normalized to unity. 
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Figure 6: Reconstructed W~ mass M^L distribution before kinematical cuts for the three tq 
signals and W + jj background at a CM energy of 500 GeV, without beam polarization. The 
cross-sections are normalized to unity. 
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Figure 7: Total transverse energy Ht distribution before kinematical cuts for the three tq 
signals and W + jj background at a CM energy of 500 GeV, without beam polarization. The 
cross-sections are normalized to unity. 




Figure 8: Charged lepton energy E\ distribution before kinematical cuts for the three tq 
signals and W + jj background at a CM energy of 500 GeV, without beam polarization. The 
cross-sections are normalized to unity. 
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Figure 9: Reconstructed top mass m\ cc distribution before kinematical cuts for the tqj signal 
and W + jjj background at a CM energy of 500 GeV, without beam polarization. The cross- 
sections are normalized to unity. 
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Figure 10: Reconstructed antitop mass m r f c distribution before kinematical cuts for the itpy 
signal and W + jj r y background at a CM energy of 500 GeV, without beam polarization. The 
cross-sections are normalized to unity. 
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Figure 11: Reconstructed W~ mass M^L distribution before kinematical cuts for the tqj 
signal and W + jj~/ background at a CM energy of 500 GeV, without beam polarization. The 
cross-sections are normalized to unity. 
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Figure 12: Reconstructed top mass m\ ec distribution before kinematical cuts for the tqZ 
signal and W + jjZ background at a CM energy of 500 GeV, without beam polarization. The 
cross-sections are normalized to unity. 
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Figure 13: Reconstructed antitop mass m ; - ec distribution before kinematical cuts for the tqZ 
signal and W + jjZ background at a CM energy of 500 GeV, without beam polarization. The 
cross-sections are normalized to unity. 




Figure 14: Reconstructed W~ mass M^L distribution before kinematical cuts for the tqZ 
signal and W + jjZ background at a CM energy of 500 GeV, without beam polarization. The 
cross-sections are normalized to unity. 
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